Unpacking the black box of improvement.

During the Salzburg Global Seminar Session 565-'Better Health Care: How do we learn about improvement?', participants discussed the need to unpack the 'black box' of improvement. The 'black box' refers to the fact that when quality improvement interventions are described or evaluated, there is a tendency to assume a simple, linear path between the intervention and the outcomes it yields. It is also assumed that it is enough to evaluate the results without understanding the process of by which the improvement took place. However, quality improvement interventions are complex, nonlinear and evolve in response to local settings. To accurately assess the effectiveness of quality improvement and disseminate the learning, there must be a greater understanding of the complexity of quality improvement work. To remain consistent with the language used in Salzburg, we refer to this as 'unpacking the black box' of improvement. To illustrate the complexity of improvement, this article introduces four quality improvement case studies. In unpacking the black box, we present and demonstrate how Cynefin framework from complexity theory can be used to categorize and evaluate quality improvement interventions. Many quality improvement projects are implemented in complex contexts, necessitating an approach defined as 'probe-sense-respond'. In this approach, teams experiment, learn and adapt their changes to their local setting. Quality improvement professionals intuitively use the probe-sense-respond approach in their work but document and evaluate their projects using language for 'simple' or 'complicated' contexts, rather than the 'complex' contexts in which they work. As a result, evaluations tend to ask 'How can we attribute outcomes to the intervention?', rather than 'What were the adaptations that took place?'. By unpacking the black box of improvement, improvers can more accurately document and describe their interventions, allowing evaluators to ask the right questions and more adequately evaluate quality improvement interventions.

Managing hundreds of improvement teams.

Recognizing the notable scale of USAID Applying Science to Strengthen and Improve Systems (ASSIST) Project activities and sizable number of improvement teams, which in some cases is close to 1,000 improvement teams managed in one country at a point in time, we sought to answer the questions: How do we manage hundreds of improvement teams in one country alone? How do we manage more than 4,000 improvement teams globally? The leaders of our improvement programs manage such efforts as though they are second-nature, without pointing to the specific skills and strategies needed to manage thousands of teams. This paper was developed to capture the lessons, considerations, and insights shared in discussions with leaders on the USAID ASSIST Project, including country Chiefs of Party and Regional Directors. More specifically, this paper seeks to describe what is involved in scaling up and managing large numbers of improvement teams. Through focus group discussions and individual interviews, participants discussed the key skills, strategies, and lessons needed to successfully manage large numbers of teams on the USAID ASSIST Project. We concluded that the six key components in managing large numbers of teams are 1) leadership; 2) management structures and capacities; 3) clear and open communication; 4) shared learning, collaboration, and support; 5) ownership, engagement, and empowerment; and 6) partnerships. We further analyzed these six components as being interrelated to one another based on the relationship between culture, strategy, and technique in implementing quality improvement activities.